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PRELIjXilNAKY REPORT ON USE OF SELF-RATINGS 
TO PROVIDE J- COEFFICIENT DATA 



INTRODUCTION 

Herb Ozur and his staff have computed the correlations and Beta weights 
for data collected by Don Wagner on five tests and six self-ratings • 

The tests (or test parts) are Spelling, Reading, Grammar, Word Meaning, 
and Filing. Self-ratings were for spelling ability, reading ability, 
grammar ability, word meaning ability, filing ability, and amount of 
education. 

Table 1 shows all the inter-correlations between test scores and ratings 



TABLE 1 

In ter correlations 



N = 20U, except for Education when N = ].97 



Read. 

Test 


Gram. 

Test 


Word 

Test 


Fil. 

Test 


Rat- 

ing 

Sj»el. 


Rat- 

ing 

Read. 


Rat- 

ing 

Gram, 


Rat- 
ing . 
Word 


Rat- 

ing 

Fil, 


Rat- 

ing 

Educa, 


spelling J 

Test 


.55 


.56 


.69 


.27 


.63 


.U5 


.hi 


.146 


.21 


• Uo ^ 


Reading 

Test 


. 


.66 


.Ih 


.3U 


■ h9 


.51 


.55 


.57 


.33 


.U9 


Grammar 

Test 


.66 


— 


.70 


.17 


.U7 


.U3 


.53 


.52 


.31 


.51 


Word 

Meaning 

Test 


.7U 


.70 




.27 


.56 


.55 


.62 


.65 


.hi 


.50 


Filing 

Test 


.3h 


.17 


.27 




.23 


.lU 


.19 


.17 


.31 


.18 


Self-Rating 
in Spelling 
Ability 


.U9 


.hi 


.56 


.23 




.61 


.67 


.61 


.Ul 


.Ul 


Self-Rating 
in Reading 
Ability 


.51 


.U3 


.55 


.lU 


.61 




.65 


.73 


.37 


.U3 


Self-Rating 
in Grammar 
Ability 


.55 


.53 


.62 


.19 


.67 


.65 




.76 


.53 


.51 


Self-Rating 
Word Abil. 


.57 


.52 


.65 


.17 


.61 


.73 


.76 


_ 


.U2 


.U9 


Self-Rating 
Filing Abil. 


.33 


.31 


.Ui 


.31 


.hi 


.37 


.53 


.U2 




.28 


Self-Rating 
in amount oi 
education 


.U9 


.51 


.50 


.18 


.Ul 


.h3 


.51 


.h9 


.28 
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Note that the rating which correlated highest with the spelling test wa 
the rating in spelling ability. The rating which correlated highest wi 
the grainmar test was the rating in grammar ability. The rating which 
correlated highest with the word meaning test was the rating in word mean- 
ing ability. The rating which correlated highest with the filing test 
was the rating in filing ability. 

Table 2 shows the Beta weights of the tests on the elements* 



TABLE 2 
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T, 

Spel. 


Read. 
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.15 
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Reading Test 


.11 


.11 


.12 


.20 
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.23 


.6^ 


Grammar Test 


.lU 


-.03 


.16 
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.29 


.62 


Word Mean. 
Test 


.16 


.03 


.12 


.32 


% 
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CD 


.19 


.72 


Filing Test 


.16 


\A 

O 

• 

t 


-.10 


.01 


On 

C\J 

• 


.11 


.35 



1 irt I* Validity Coefficients 



J^ow J- Coefficients Might Be Derived 

Suppose that it is decided that for a particular Job^ either usu'np 
job element blank or other analytic method, that the job criterion ox 
success would be as follows: 

Element Weight 

Education h 

Reading 10 

In traditional validation studies, it would be necessary to rate people 
in education and reading on the job and derive a criterion, administer 
tests, and secure validity coefficients. This is not necessary with the 
J-Coeff icient technique. There are only three simple steps, once tables 
like the above are available. 
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STEP 1 



Compute the standards deviation of the composite for standard scores 
for education and reading. In standard score form, the standard 
deviation of education would be h and of reading would be 10. The 
standard deviation of the composite is the square root of the s\m of 
the squares of the standard deviations plus twice the product of the 
intercorrelation times standard deviations, in this case: 



\/h^ + 10 ^ + 2 (.li3) {h X 10 ) = 12.261; 

The .U3 came from Table 1, the correlation between education and reading. 
STEP 2 



We need the correlations of each -element with the composite criterion - 
The numerator of each correlation is the sum of the products of the 
weight for the selected elements times the correlation of the particular 
element with the selected element. 

The denominator of each dorrelation is the standard deviation oi the 
composite, found in Step 1. Note that the standard deviation ol each 
element is one, since elements are regarded to be in standard score fornn. 
Therefore, the standard deviation for each individual element can be 
ignored in the element. For example, the correlation between tb 
for spelling ability and the composit e ^ ^ured as followr: 



(.61 X 10) + (.Ul X U) = .6311 
12.26U 



Following are the correlations between the rating on each element and the 
composita criteri ^n: 



Spelling ability 


.6311 


Read in g ab il i ty 


.9600 


Grammar ab ili ty 


.6963 


Word meaning 




ab ility 


.7550 


Filing ability 


.3930 


Education 


.6768 



Note that for J-Coeff icient purposes, diagonal entries in Table 1 are 
unity; lack of reliability has already influenced the Beta weights, 
insofar as relation of xest to job is concerned. (See page 5) 




2j/lf there are more than two elements, 
one for each correlation. 



there are 



n(n-l) produc ts, 
2 



4 



1 ^ 



STEP 3 — Finding the J- Coefficients 

The J-Coefficients are sums of products of the test Beta weight on 
each element times the correlation of each element with the composite 
criterion. For example, the J- Coefficient for the spelling test is 
the sirni of the following products: 



Correlation of Element 



Element 


Beta Weight From 


With Composite 




Table 2 


Computed Above 


Spelling 


-31i 


.6311 


Reading ability 


.Oil 


.9600 


Grammar ability 


.01 


.6963 


Word meaning ability 


.06 


.7330 


Filing ability 


-.10 


.3930 


Education 


.13 


.6768 



The sujri of the products — x 6311^ .Oli x 9600, etc., is .Ii9li0. 
Tt^ ■ 3 is the J-Coefficient for the spelling test. 

The J- Coefficients for all the tests are as follows: 

Test J-Coeff icient 

Spelling .Ii9li 

Reading .381 

Grammar -311 

Word Meaning .613 

Filing .179 
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Test Coverage By Elements 

The formula for the J-Coeff icient , as shown in Basic Formulae for the 
J-Coeff icient (1955), has^^^in the numerator, and R+ R • in the 
denominator. r ^ 

The denominator is the produce of the multiple R of jobs on the elements, 
times the multiple R of tests on the elements* 

The following paragraph on pages 7-8 of Basic Formulae for the J-Cosff icient 
deals with this denominator in the final formula (formula 17): 

"With respect to the denominator of (17), it is not necessary 
that the checklist should provide enough elements to cover 
all test and job variances. It is enough to know that the 
multiple R of a complete set of elements with test or with 
criterion is 1. However, if some job-test covariance is not 
covered in the elements, that is, if the checklist is in- 
complete in some abilities present in both job and test, 
then the numerator of (1?) will not be completely represented 
by data. This incompleteness may be adjusted in at least two 
ways, as it becomes apparent in validity studies. An obvious 
^ way to increase the coverage of the checklist is by adding 

new elements. An example will be given later. If, however, 
it appears for a certain test or kind of job that the job-test 
covariance for elements present in the list is, although in- 
catipleto, proportional to total covariance (that is, that 
calculated J-coefficients are a multiple of validity coefficients), 
then adjustment may be made for a test or kind of job by a 
compensating factor." 

Note that any correction in this regard would raise the J-Coeff icient. 

Thus, the J-Coeff icient is a minimum bound of the validity coefficient. 

/ 

If, for a particular test, and set of elements, J-Coeff icients are 
obviously too small because of inadequate number of elements, the best 
answer would be to add elements so that the R's will be closer to 1. 

Another alternative would be to place the R of the test in the denominator, 
as indicated by the formula. This would raise the J-Coeff icient, implying 
that relationship between job and test on udcnown elements is the same 
as the relationship for known elements. Before placing R in the denominator, 
unreliabilxty of test would be considered as an element not- correlated 
with other elements. According to standard test theory, the Beta weight 
and correlation of unreliability with the test would be the square root 
of unity minus the square of the reliability coefficient. Since the multiple 
R is the square root of Beta r products, the multiple R of test on all 
elements plus unreliability is the square root of r2 plus (1 - r^), where 
R is the multiple R of test on elements and r is reliability of test. 
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With respect Df R for job, there is no statistical problem, since the 
composite criterion is made up of the given elements by the same 
kind of decision as in any validity study. If additional elements 
are needed to define the job, they should be added to the list, rather 
than compensated by increasing . the J-'Coeff icient to provide for their 
absence. 



Part II. Use of Coefficients to Analyze Tests 

Before the J-Coeff icient was available, it was thought necessary that 
analytical factors be uncorrelated, sc that their products could be 
suinmed to equal test intercorrelations . This required that factors 
be uncorrelated. 

As Thurstone pointed out, elements in real life are not uncorrelated. 

For e^omple, he pointed out that the dimensions of boxes are correlated. 
Boxes with large widths tend to have large lengths. The need for orthogonal 
factors resulted in artifacts. 



The J“ Coefficient formula makes it possible to deal with elements that are 
correlated. Thus, the procedure begins with elements that can be under- 
stood by raters, and the weights are based on the meaning given these ^ 
elements by raters. (Of course, if elements are independent. Beta weights 
and r*s are identical, and such elements would be orthogonal, with load- 
ings equal to Betas and to r*s.) 

The discussion of the table on page 2 illustrates some of the insights 
that can be gained by J-Coeff icient analysis of the relation between 
tests and elements. Further analysis can be done by comparing actual 
test intercorrelations with J-Coeff icients between tests. 



When validity coefficients were computed, reliability of tests was 
not considered because unreliability of a test was lumped with other 
elements in the test that were not in the job. 



With tests, however, there are elements that are not in the Job elements, 
and to analyze these elements, it is necessary to subtract out un- 
reliability. 



For the spelling, reading, grammar and word meaning tests, the Kuder- 
Richardson formula based on number of items, standard deviation and^ 
mean was used, since these tests are power tests. The filing test is 
a speed test. For this test, a reliabTlity^ formula developed by Primoff 
which meapares each response in the test with all other responses, and 
derives a reliability coefficient for a hypothetical set of measures, 
was used. Following are the reliability coefficients: 

Test Reliability Coefficient 




Spelling 

Reading 

Grammar 

Word Meaning 
Filing 



.79 

.'Io5 



•7 



*7 



in test theory, 'onreliability of the test is considered an element un- 
correlated with ary other element-, and correlated only with the par- 
ticular test. Thus, the Beta and the r for unreliability are identical. 
If unreliability is considered to be an element. Table 2 would have an 
additional entry, unreliability, with a Beta weight vr, and Table 1 would 
have an entry for unreliability with an r equal to w. Now the multiple 
R shown in Table 2 would be greater. The product for the Beta times r 
for unreliability would have to be added to the other products of Beta 
times r to give the augmented R, considering unreliability as an element. 

the reliability of the test can be considered to be the cor- 
relation and Beta weight of the test on a single "reliable" element. 

Thus, the square of reliability plus the square of unreliability equals 
‘One. Thus, both the Beta and the r for unreliability are: 



= r ^ 1 - reliability^ 

The augmented multiple R for a test including unreliability as an element 
will be the sum of all Beta^r-prodpctduc ^ represents the Beta-r products 
for the elements in Table 2. The additional product for unreliability will 
be one minus the square of the reliability coefficient. (Multiplying 
Beta for unreliability by r for unreliability.) Thus, 

2 

(Augmented R) = 1 - reliability^ + (original R)^ 

The f shows the test interrelationships: j _ 

.i— , g Au^ented R Rroduct 

^-»t^is the sum of the products of test Beta weight on 
element timss test correlation on element. For example, 
for the spelling and reading tests, the Beta for the 
spelling test on Element 1, is multiplied by the r 

of the reading test on Element 1, .U9; the Beta .Olt is 
multiplied by the r .^1, the Beta .01 by the r .55, the 
Beta .06 by the r .57, the Beta minus .10 by the r .33, 
and the Beta .l5 by the r .Lt9. The products are summed 
to give ^ ^ 

(Except for rounding errors, an identical sum would bs 
obtained by using Betas for the reading test and r for 
the spelling test.) 

To obtain J-Coeff icients between tests, each s\am of Beta r products is 
divided. by the product of the augmented R's. 



O 



Table 3 compares the J-Coeff icients between tests with actual test inter- 
correlations. Test intercorrelations are shown in Table 1, but in no way 
affec-ted the Beta weights or r's between tests and elements which were 
used in computing J- Coefficients. 
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TABLE 3 

J- Coefficients Between Tests Compare^ to Actus., Correlations 







Aug- 




Actual 


Test 




men- 


J- Co- 


Inter- 


■B 'p-',- 


ted 

R 


efficient 


correlation 


Spelling Test vs. 




.90 , 






Fciading Te 


- 37 


r . ' J 

• y ^ * 


.hi 


.55 


Grammar Test 


.35 


• 95 i 


.hi 


.56 


Word Mean. Test 


.U1 


.81 


.51 


,69 


Filing Test 


.lit 


.87 


.18 


.27 


Reading Test vs. 




.93 






Spelling Test 


.37 


.90 


.hh 


.55 


Grammar Te st 


.39 


.95 


.U5 


.66 


Word Mean.Test 


. Ii6 


• 8U 


.59 


.7U 


Filing Test 


.15 


.87 


.19 


.3U 


Grammar Test vs. 




.95 






Spelling Test 


.35 


.90 


• Ul 


.56 


Reading Test 


.39 


.93 


.U5 


, 66 


Word Mean. Test 


. IiU 


.8U 


.55 


.70 


Filing Test 


.15 


.87 


.18 


.17 


Word Mean# Test vs. 




• 8U 






Spelling Test 


.hi 


.90 


.5U 


.69 


Reading Test 


. It6 


.93 


.59 


.7h 


Grammar Test 


. liU 


.95 


.55 


.70 


Filing Test 


.18 


.87 


.25 


.2? 


Filing Test vs. 




oo 






Spelling Test 


.Ih 


• 90 


.18 


.27 


Reading Test 


.15 


.93 


.19 


.3h 


Grammar Test 


.15 


.95 


.18 


.17 


Word Mean. Test 


.18 


.8U 


.25 


.27 
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ne fxrst col™n xn Table 3 lists the tests bein^ ''ompared. The second 
column gxves ^or each interrelation. The col'xinn shows the 

augmented R, computed as explained above on the ba./Is of the R in Table 2 

the J- Coefficient 

computed by dxvxdxng each by the product of the augmented R's for 

the tests bexng compared. For exx'-; le, for the Spelling Test vs. the 
H6a.ctiLng Tsst^ j _ *37 

.90 X .93 "" 

actual : -.r correlation between the tests, as 
g Table 1. As nc.ed before, these intercorrelations had not been 

used xn calculatxng F ’s or J's. Aside from the fact that correi^ndence 
between J-Coeff xcxents for tests and actual intercorrelations of the tests 
xs a cneck of the operation of the J-Coeff icient formula, the comparison 
between J's and xntercorrelation is an analytical device. 

correspondence in relative order between both the unmodified 
r or the J-Coeff xcients and the original inter correlations. The modifica- 

^ affected the ranks very little, but did affect means and 
standard deviations of the ineasiiresj 



.9696 
• 97li3 



r between J-Coeff icient and actual intercorrelations 
r between and actual intercorrelations 

Mean, actual intercorrelations .1x9^ 

Mean, J-Coeff icients .378 

Mean, "jf? ^ . 301 ^ 

Standard deviation, actual intercorrelations .201Ii 

Standard deviation, J—Coeff icients .1332 

Standard deviation, >=T 7 .1236 

X considered along with Tables 1 and 2. It can be seen in 

Table 3 that the elements _ listed in Table 2 practically determine the inter- 
correlatxon between the Filing Test and the Grammar Test ( J-Coeff icient I 8 
actual xntercorrelation .17), and the Word Meaning Test ( J- Coefficient 23 ’ 

actual xntercorrelation .27). On the other hand, there are other elements 
coimon to the Fxlxng Test and the Spelling Test ( J-Coeff icxent .18, actual 
xntercorrelatxon .2?), and the Filing Test and the Reading Test ( J-Coeff icient 
. 19 , actual xntercorrelation ,3h). Similarly, the extent of common coverage 
y the given set of elements can be seen for other combinations of tests. 



this type of analysis is that the elements are elements 
that can be understood by people in general, since the Beta weights were 

determxned by correlations with the elements as understood by people in 
general. t' t' 

This kind of informat xon is obviously useful in developing new tests, 
xmproving old tests, and looking for the actual meaningful elements that 
make up tests. 
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Part TII - Use of J~ Coefficient Analysis in the Job Element Procedure 



The original development of the J-'Coefricient procedure, released in 
19?^^ required replications of validity studies, which made it im- 
practical to develop test Betas for other than a specified set of elements. 
The use of self-ratings to obtain test Betas makes it possible to analyze 
tests in terms of the most appropriate elements for a particuH.ar set of 
jobs, and thus makes J-Coeff icient analysis appropriate for all job-ele- 
ment examining - 

With the job-element procedure, specific elements are chosen for par- 
ticular jobs. Ability in the elements may be measiired by all available 
evidences, one evidence being a written test. Other evidences include 
past accomplisimients and training. 

Exact Criterion in Terms of Job-Elements 



The first question is the validity of a test for a given requirement 
in terms of a particular job-element pattern. 

In Part I of the present report, it was assumed that the criterion is 
simply the sum of abilities in elements, weighted for a particular job. 
This is inexact, since the elements correlate. 



For example, if a criterion should include ability in elements A, B, and C, 
with unit weights applied to the standardized rating in each element, and 
if elements A and B correlate .80 together, but 0 with element C, the 
correlation between element A and the criterion will be (letting a, b, and 
c be standardized ratings in elements A, B and C) ; 



A/ 



{ ^ -f J} y 



Since the ratings are standardized. 












Also, the problem states that r is .80, while r = r =0. 

AB AC BC 

Making these substitutions and bringing N to the mmierator, the above 
expression becomes: 



N 



V ^ ^ 0 
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Since ratings are standardized. 



<r^- A 
' 7 T ^ ^ 

^ ^ ^ JT • 

—-77 ^ ^ 

Making these substitution s^ the above expression for the correlation 

between Element A and the criterion is I.80 01 

= . ou 

\/tt:eo 

On the other hand, the correlation between ability C and the criterion ^ 

will be dependent on -j- i> — g" C- V~ €. S <Z ^ J 









c: - e 0 . 

'~~f^ ^ ^ tAf ^ ^ J ' 



(T^ ^ ^ 












Thus, ability C will correlate with the criterion only 



The relative importance of an element in a criterion may be measured by 
the square of its correlation with the criteria)^. This is because the 
square of the correlation is the proportion of variance in the criterion 
that is calculable by regression from the particular element. - If elements 
were independent, the squares of the correlations would add to unity, and 
be obvious proportional contributions to variance. When elements are not 
independent, the squares of the correlations still represent the relative 
extent of independent contribution of each element to the criterion. 

Thus, although Elements A, B, and C were given equal unit weights in de- 
termining the criterion. Element A (and similarly Element B) has a 
relative importance of .8U^ = 71 in the criterion, while Element C has 
a relative importance of only = .22 in the criterion, only 31 ^ as 

much. 

This example demonstrates that when elements are combined into a criterion, 
those elements which have the greatest general communalities withthi^he/^ 
elements obtain greater importance than the weight being applied. 

For greatest accuracy, which is important in fair testing, each element 
should contribute to the total criterion in proportion to its importance 
as determined by careful analysis. 
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Example of an Exact Criterion in Terms of Job Elements 

For example_, suppose that a simple mail and Tile clerk 30 b has been 
analyzed^ and the consensus or informed opinion is that overall ability 
in the job is made up of the following pattern: 

Element Estimated Importance 



Spelling 

Ability 3 

Reading 

Ability 2 

Ability to Use 

Grammar 0 

Understanding Word 

Meaning 1 

Filing Ability 10 

Education 2 



The importance values in the second column may be obtained as an average 
of judgments using one of the Job analysis procedures of the job-element 
method. By evaluating elements in terms of importance^ judges usually 
mean that the elements should be represented in the criterion in the same 
relative proportions. They mean that spelling ability is three times 
as important as understanding of word meaning, etc. They do not usually 
mean that the criterion is determined by adding three parts of spelling 
ability to two parts of reading ability, etc. Statistically, this implies 
that the estimated importances express the desirable influence that each 
ability should have in a cr iter ior^ the proportion that each ability would 
contribute if all the abilities were independent 

The ideal criterion would then be one in which the Importance of each 
ability in criterion variance would be in proportion to its estimated 
importance. That is, if each ability is correlated with the criterion, 
the squares of the correlation? would be in the proportions of the above 
estimated importances. 

The question then becomes: What are the weights to be applied to 
standardized ratings in the elements which will result in each element 
having an influence on the criterion in proportion to the estimated 
importances? 
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Let such weights for an exact criterion oe S for spelling aoiJ.2.ty, H. lor 
reading ability, G for ability to use gramiriar, W for understanding of 
word meaning, F for filing ability, and E for education. 



If s, r, g, etc. are standardized ratings in each element, the correlat on 
between the element spelling ability and the crite^on_ equals; 






J ^ 









■S^ 



JV/ 



S£T 



: 






f^Q- /Tfc^ 



This becomes: 



-t 







N 



^ ^ U/ -/ 



N 






V 









I. 

-N ' £ 



hi 



FS 






S + R^„ - 












This is the correlation between spelling ability and the criterion. 
Similar expressions can be found for the correlation of each Oj. the other 
elements with the criterion. 
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The identical denominator would appear in each correlation, since uhe 
denominator is j|. times the product of the standard deviation of the 
element, which is 1 since ratings are standardized, times the standard 
deviation of the criterion which is the porti^ xinder the square root 
sign. 

Sets of simultaneous equations may then be developed, proportions of squared 
correlations being set equal to proportions of estimated importances. 

For example, if r is the correlation between spelling ability and the 
criterion, and r is the correlation between reading ability 

and the criterion, then 




should be 3 



iji proportion to the importances of spelling ability and of reading ability 
on page 12 , so that 



= yT“ 
R 

RC 



Similarly, if r 

WC 



is the correlation between understanding word meaning and 



the criterion, then - 



2 

r 

RC 



r 



2 



WC 



£ and 
1 







There are six unknowns - S, R, G, ¥, F, and E. Therefore, we need six 
linear independent equations. These may be set up in any convenient order, 
proportioning two correlations at a time, and equating to the square r^ot 
of the proportions of estimated importance. For the various intercorrelations, 
the intercorrelations among ratings shown for the elements in Table 1 are 

used. (For example, r is .61; r is .67, etc.) In each proportion, the 

SR SG 



denominators of the correlations are ignored since denominators cancel when 
correlation is divided into another. 



one 



Following are six ec^^ai:ions which might be set up^ following the 
generalized formula for a correlation between a single element an 
the criterion on page 13 . 



eg. 1 ^ 



based on 




-%C 




1.225 




S + .614? + .67G + .61W + . LXF + .LIE = 1,225 
R + .615 + .65G + .73W + . 37F + .IJE 



eg. 2 j based on j r 



RC _ 






V/2- = 1 




R + .615 + .65g + .73W + .37F + .U3E ^ i.Mb 

W + . 61S + . 7 3R + . 7 6G + . L.2F + . L9E 



eg. 3> based on estimat.ed importance of grammar to be 0 so that 

r =0: 

GC 

G + .675 + .65R + .76W + .53F + .5lE = 0 



eg. based oi 




VTO = 3.163 




F + .UlS + .37R + .53G + .U2W + ,28E = 3.163 
W + .615 + .73R + .76G + .L2F + .L9E 



eg. 5, based on /r 

L FC V/^ 



.UU7: 



1 



E + .UlS + .U3R + .5IG + .U9W + .28F = .UU7 
F + .UlS + .37R + .53G + .U2W + .28E 



O 
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The last equation^ equation 6 ^ is defined by the expression which 
represents the variance of the criterion_, in the denorr^inator of tne 
corre^.ation between element and criterion (for example, between 
spelling and criterion on page 13). In order that each numerator 
actually be a correlation coefficient, it would be necessary that 
this variance be 1. 

Thus, equation 6 is: 

2 2 9 2 2 

S^ + R + G + W + F + E + 2 (.61SR + .67SG + .61 SW + . UlSF + .l4lSE 
+ .65RG + .73RW + .37RF + .U3RE + .76GW + .53GF + .$1GE + .1^2 WF + 

+ .28FE) = 1 

For hand operation, it is probably easiest to solve first for S using 
equation no. 1; substituting this in equation 2 and solving for R, etc. 

It is desirable at each step to check the partial answers in the 
original equations, substituting 1 for the letters still unknown. 

Because the expression in equation 6 was set equal to 1, each numerator 
and each denominator in equations 1 to 5 represents a correlation of an 
element with the criterion. After the simultaneous equations are solved, 
the correlations are found to be: 

Correlation with 



Element Criterion 

Spelling ability .377 

Reading ability .308 

Ability to use grammar .000 

Understanding word meaning .217 

Filing ability .687 

Education .307 



Finding the J~Coef ficients for the Exact Criterion 

Now, using the Beta weights for tests on the elements which have been 
given in Table 2, multiplying by the above correlations to get cross 
products, and summing the cross products provides the following J-Co- 
efficients: 
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J- Coef 1 icierit 
( Validity") 



Word Meaning Test 
Filing Test 



Spelling Test 
Reading Test 
Grammar Test 



.206 
.217 
• 185 
.252 
.280 



The highest J-Coeff icient, as one might expect, is for the Filing Test. 

Remember that the job analysis indicated that the element Filing ability 
was to count most. If we wanted only the test that has the highest 
validity for filing ability, regardless of other abilities. Table 1 shows 
that the test with the highest correlation for Filing ability would be 
the Word Meaning Test, with correlation of •I 4 I. The Filing Test had a 
correlation of only .31 with the Element Filing ability, less even than the 
Reading Test which was .33. However, because the Word Meaning Test 
also taps irrelevant abilities to an unnecessary extent, the above J'-Co- 
eff icients show that it would correlate with a fair criterion of exactly 
proportioned content only .252, while the Filing Test has the highest 
J-Coef ficient , •280 with this criterion. In order to get a high score 
on the Word Meaning Test, ‘an applicant must be highly qualified on ele- 
ments unrelated to the job. When a criterion is set up in which the 
elements are emphasized in terms of their actual importance on the job, 
the Word Meaning Test correlates with this criterion lower than it would 
with Filing ability alone. 

Other Indications of Ability 

The Job-Element Procedure permits all evidences to be used to evaluate ap- 
plicants. In the present experiment, there was only a limited amount of 
non-test evidence, but for explanatory purposes this evidence will be 
explored . 

In the present examination, applicants were asked to tell whether they had 
a typing or stenography job where filing was important. One might expect 
persons who claim high proficiency in filing, especially if they have had 
relevant experience, to be worthy of special consideration in evaluating 
the elements, even if they do not achieve a high score on the filing test. 
Such special consideration might be checking with previous supervisors 
where they had related employment, or use of performance tests where they 
did not. Applicants who demonstrate ability on the elements through 
non-test evidences may be considered qualified along with those who 
achieve high test scores. 

The following table illustrates some of the features of the problem. It 
shows cumulative frequencies for test scores for five group s^ Total 
competitors (20h); competitors who claimed to be highly qualified in 
filing ability and who claimed filing experience (30); competitors who 
claimed to be highly qualified in filing ability but without experience ( U2) 
competitors who claimed to be experienced but not highly qualified m 
filing ability (3U); and the remainder (98). 
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n:LT.-ilat. ive Frequencies on. ruling: 





18 



15 



Note that applicants in col'i-jnns 2 and 3 d^ have better test resu-^t.s nnan 
others^ as may be expected. 

The mean test score was 13 #01’^ the standard deviation 5-72. ir we expect 
to interview for employment all who make at least ^ ^TV i^he mean 

(upper quantile of normal distribution), we wouldY3 include all with a 
test score of 17 or over. 

In column 2 are 18 applicants with test scores below 17 but who claojn 
filing experience and a high degree of filing ability. Those of tnis 
group whose claims are verified by checks with employers could be added 
to the $0 who received a test score of 1? or over. 

If practicable, applicants in columns 2 and 3 with less than a score oi 17 
on the test, for whom an employer check is not feasible, might be given a 
performance test in ability to file. If a performance test is used, the 
resulting score could be scaled similarly to the Filing Test. 

Part IV - Derivation of J~Coeff icient in Terms of Elementary Algebra 

The derivation of the J-Coef f icient published in 1955 involved calculus 
and determinants. It is characteristic of scientific formulas originally 
derived by calculus that as their use becomes better understood, it oecomes 
possible to develop derivations in terms of algebra. The equations in 
Part III suggest a development which requires no more than algebraic 
manipulation of summed products. 

Let: 

A., B , C-; — be standardized ratings of applicant i on Element 8 A*, BSC* — 
X i ' ^ 



a, b, m — — be tentative weights anolied to the . standardi*€id scores A, B,C — 
on Silent s AU B' *C *—^t^ det^mine a score in Test T. 

■ be a standardized score on a criterion for applicant i. 



The correlation between test and criterion will be (where, is the S.D, of the 

derived composite test score) - - 

^ ( ^. A 6 3 oj A/ 



J-Coeff icient 






^ 3)C ^ 






Ai 






A 



r^/ 






a Y 

where -jj — is the 



correlation between Element A* and the criterion, the other two terms in the nvun— 
erator are similarly correlations between Elements B' and C and the criterion, and 
^ is 1, TSms7'lsit*”Jii«beT^ betwee* Test T and criterion X is: 

^^:A«xfbrB,y» crciy 



I 



A special elemer.t is assurr.ed for the test — an error element^ having no 
correlation with any other element or with the criterion. As in general 
test theory, with a set or standari zed test scores , the correlation between 
fcest score 'and error element isl/l^reliability^. This would not affect 
the numerator, because the product of the weight for error, times the 
correlation of error with criterion, is 0, since the correlation of error 
with criterion is 0. On the other hand, the error element does affect 
since the square of its weight adds into (Since error is independent o£ 

other elements. Beta weight and r for error element on test are egual). 

Let eac i of the tentai:,ive test weights a, b, c — be divided by ^ so 
that the adjusted weights are a b £ etc. 

Now, bringing ^S^into the ninnerator: 



'^TX ” 



r. 



lY 



^ ^rc 



Thus, the J-Coeff icient is the sum of the products of adjusted test weight 
in each element times the correlation of the element with the criterion# 

Where the test weights completely determine the test scores, except for 
unreliability, 

a , b , etc. will be test Beta weights on the elements, 
with the situation that the augmented R as described on page 7 is unity. 



